Detection In Time Series Data Research Articles

Spatial-Temporal Graph Deviation Neural Networks for Anomaly Detection in Time Series

Abstract Anomaly detection in time series data (e.g., sensor data) is becoming a fundamental research problem that has various applications. Due to the complex inter-sensor relationships, it is challenging to detect anomalous events such as system faults and attacks hidden the high-dimensional time series. Recent advancements in deep learning approaches such as Graph Neural Networks (GNN) have greatly improved anomaly detection performance in time series data. However, existing methods usually use separate components to capture spatial and temporal dependence relationship and ignore the heterogeneities in spatial-temporal data which motivates us to capture them together. In this paper, we propose a novel approach STGDNN (short for Spatial-Temporal Graph Deviation Neural Networks) that learns the spatial-temporal dependence relationship together in structure learning of graph neural networks. In addition, we use the learned spatial-temporal graph structure and attention weights to explain the detected anomalies. The experiments on three real world datasets show our superiority in detection accuracy, anomaly diagnosis, and model interpretation compared with state-of-the-art methods.

Time-domain event detection using single-instruction, multiple-thread gpGPU architectures in single-molecule biophysical data

Discrete amplitude levels in ordered, time-domain data often represent different underlying latent states of the system that is being interrogated. Analysis and feature extraction from these data sets generally require considering the order of each individual point; this approach cannot take advantage of contemporary general-purpose graphics processing units (gpGPU) and single-instruction multiple-data (SIMD) instruction set architectures. Two sources of such data from single-molecule biological measurements are nanopores and single-molecule field effect transistor (smFET) nanotube devices; both generate streams of time-ordered current or voltage data, typically sampled near 1 MS/s, with run times of minutes, yielding terabyte-scale datasets. Here, we present three gpGPU-based algorithms to overcome limitations associated with serial event detection in time series data, resulting in a 250× improvement in the rate with which we can detect salient features in nanopore and smFET datasets. The code is freely available.

Anomaly detection in time-series data using evolutionary neural architecture search with non-differentiable functions

Deep neural networks have become the benchmark in diverse fields such as energy consumption forecasting, speech recognition, and anomaly detection, owing to their ability to efficiently process and analyse data. However, they face challenges in managing the complexity and variability in time series data, often leading to increased model complexity and prolonged search duration during parameter tuning. This paper proposes a novel anomaly detection approach through evolutionary neural architecture search (AD-ENAS), which is specifically designed for time series data. The proposed approach focuses on the search for the optimal and minimal neural network architecture. The AD-ENAS method consists of two main phases: architecture evolution and weight adjustment. The architecture evolution phase highlights the importance of neural network architecture by evaluating the fitness of each network agent using shared weight values. Subsequently, the convolutional matrix adaptation technique is used in the next phase for optimal weight adjustment of the neural network. The proposed AD-ENAS method operates without relying on differentiable functions, thus expanding the scope of neural network design beyond traditional backpropagation-based approaches. Various non-differentiable loss functions are explored to facilitate effective architecture search and weight adjustment. Comparative experiments are conducted with five baseline anomaly detection methods on three well-known datasets from reputable sources such as NASA SMAP, NASA MSL and Yahoo S5-A1. The results demonstrate that the AD-ENAS approach effectively evolves neural network architectures, outperforming baseline methods with F1 scores across the three datasets (MSL: 0.942, SMAP: 0.961, Yahoo S5-A1: 0.988) with non-differentiable loss functions, showcasing its efficacy in detecting anomalies in time series data.

Exploiting negative correlation for unsupervised anomaly detection in contaminated time series

Anomaly detection in time series data is crucial for many fields such as healthcare, meteorology, and industrial fault detection. However, traditional unsupervised time series anomaly detection methods suffer from biased anomaly measurement under contaminated training data. Most of existing methods employ hard strategies for contamination calibration by assigning pseudo-label to training data. These hard strategies rely on threshold selection and result in suboptimal performance. To address this problem, in this paper, we propose a novel unsupervised anomaly detection framework for contaminated time series (NegCo), which builds an effective soft contamination calibration strategy by exploiting the observed negative correlation between semantic representation and anomaly detection inherent within the autoencoder framework. We innovatively redefine anomaly detection in data contamination scenarios as an optimization problem rooted in this negative correlation. To model this negative correlation, we introduce a dual construct: morphological similarity captures semantic distinctions relevant to normality, while reconstruction consistency quantifies deviations indicative of anomalies. Firstly, the morphological similarity is effectively measured based on the representative normal samples generated from the center of the learned Gaussian distribution. Then, an anomaly measurement calibration loss function is designed based on negative correlation between morphological similarity and reconstruction consistency, to calibrate the biased anomaly measurement caused by contaminated samples. Extensive experiments on various time series datasets show that the proposed NegCo outperforms state-of-the-art baselines, achieving an improvement of 6.2% to 26.8% in Area Under the Receiver Operating Characteristics (AUROC) scores, particularly in scenarios with heavily contaminated training data.

Out-of-Distribution Detection in Dependent Data for Cyber-Physical Systems with Conformal Guarantees

Uncertainty in the predictions of learning-enabled components hinders their deployment in safety-critical cyber-physical systems (CPS). A shift from the training distribution of a learning-enabled component (LEC) is one source of uncertainty in the LEC’s predictions. Detection of this shift or out-of-distribution (OOD) detection on individual datapoints has therefore gained attention recently. But in many applications, inputs to CPS form a temporal sequence. Existing techniques for OOD detection in time-series data for CPS either do not exploit temporal relationships in the sequence or do not provide any guarantees on detection. We propose using deviation from the in-distribution temporal equivariance as the non-conformity measure in conformal anomaly detection framework for OOD detection in time-series data for CPS. Computing independent predictions from multiple conformal detectors based on the proposed measure and combining these predictions by Fisher’s method leads to the proposed detector CODiT with bounded false alarms. CODiT performs OOD detection on fixed-length windows of consecutive time-series datapoints by using Fisher value of the input window. We further propose performing OOD detection on real-time time-series traces of variable lengths with bounded false alarms. This can be done by using CODiT to compute Fisher values of the sliding windows in the input trace and combining these values by a merging function. Merging functions such as Harmonic Mean, Arithmetic Mean, Geometric Mean, and Bonferroni Method, etc. can be used to combine Fisher values of the sliding windows in the input trace, and the combined value can be used for OOD detection on the trace with bounded false alarm rate guarantees. We illustrate the efficacy of CODiT by achieving state-of-the-art results in two case studies for OOD detection on fixed-length windows. The first one is on an autonomous driving system with perception (or vision) LEC. The second case study is on a medical CPS for walking pattern or GAIT analysis where physiological (non-vision) data is collected with force-sensitive resistors attached to the subject’s body. For OOD detection on variable length traces, we consider the same case studies on the autonomous driving system and medical CPS for GAIT analysis. We report our results with four merging functions on the Fisher values computed by CODiT on the sliding windows of the input trace. We also compare the false alarm rate guarantees by these four merging functions in the autonomous driving system case study. Code, data, and trained models are available at https://github.com/kaustubhsridhar/time-series-OOD.

Markov enhanced I-LSTM approach for effective anomaly detection for time series sensor data

Users could engage and interact with their immediate surroundings without effort in smart settings. The emergence of intelligent technologies along with software-based services has made this possible. It is clear that technical advancements have ushered in a new era for both computer processing and sensor technology, facilitating the concept of smart surroundings. Even though their implementation faces a number of obstacles, numerous expansive projects are working to advance their adoption. The problem of anomalies in the sensor data could result inappropriate decisions and could lead unamicable situations to the users. Many such algorithms are already there, which does not provide satisfactory predictions for the sensor data for the time series data. Time series anomaly detection problems are typically stated as finding outlier data points in comparison to some norm or typical signal. Better anomaly detection in time series data is provided by the proposed Markov and enhanced LSTM technique. The Markov model and the enhanced LSTM offer accurate predictions for extra- and short-term data, which is highly useful in situations involving intelligent environments. When compared to the KNN algorithm, the technique offers reduced MAE, RMSE, MSE and MAPE errors. The algorithm also performs better than other LSTM and RNN methods. The proposed algorithm provides 0.00047 reduced error in humidity data, 0.00416 reduced error in temperature and 0.01771 reduced MAE value in case of light intensity when comparing with the KNN algorithm.

Developing a constraint model for using artificial intelligence on existing, limited hardware in manufacturing machines

In the rapidly evolving field of Artificial Intelligence (AI), its application in industrial settings, particularly for anomaly detection in time series data, poses unique challenges. Current methods often lack a comprehensive understanding of the trade-offs involved in achieving optimal model performance, data preparation effort, and prediction quality. To bridge this gap, this paper presents an adaptive approach to address these challenges, focusing on making conscious decisions about mentioned trade-offs. Inspired by the principles of the Iron Triangle from product engineering, our methodology introduces a novel ”AI triangle” with dimensions of Speed, Effort, and Quality. We applied this methodology to a real-world case study involving anomaly detection in a constrained hardware environment in the context of a forming production process. The results highlight the effectiveness of our approach in achieving a practical balance between speed, effort, and quality constraints for implementing AI in an industrial setting. This paper provides valuable insights into the considerations and trade-offs necessary for the successful deployment of AI in manufacturing and other similar industrial applications.

Identification of stock market manipulation using a hybrid ensemble approach

Anomaly detection in time series data is a complex data mining issue with many useful, real-world applications. Anomalies in datasets represent deviations in the expected behaviour of a system and can indicate rare but significant events that require intervention. Market manipulation is a serious issue in financial jurisdictions worldwide, with financial regulators such as the SEC constantly trying to prevent it and prosecute those guilty of it. This paper makes use of state-of-the-art deep learning techniques as well as more classical statistical techniques in order to detect anomalies in five real-world datasets. The predictions of these models are then aggregated in two different ensemble models. The results of the individual models as well as the ensemble models, are evaluated, and F1-Score measures performance. Nine individual models, consisting of three models based on LSTM with Dynamic Thresholding, three ARIMA models and three Exponential Smoothing models, were used to generate predictions of anomalies based on daily trading volumes. The individual predictions of these models were then aggregated, with two different ensemble methods being used, namely the majority voting ensemble method and the ensemble averaging aggregation method. While both performed well, the majority voting ensemble method was seen to be the superior method in this study, with an average F1Score of 0.494, compared to an F1Score of 0.414 for the ensemble averaging aggregation method.

An Experimental Evaluation of Anomaly Detection in Time Series

Anomaly detection in time series data has been studied for decades in both statistics and computer science. Various algorithms have been proposed for different scenarios, such as fraud detection, environmental monitoring, manufacturing, and healthcare. However, there is a lack of comparative evaluation of these state-of-the-art approaches, especially in the same test environment and with the same benchmark, making it difficult for users to select an appropriate method for real-world applications. In this paper, we present a taxonomy of anomaly detection methods based on the main features, i.e., data dimension, processing technique, and anomaly type and six inner classes. We perform systematic intra- and inter-class comparisons of seventeen state-of-the-art algorithms on real and synthetic datasets with a point metric commonly used in classification problems and a range metric specifically designed for subsequence anomalies in time series data. We analyze the properties of these algorithms and test them in terms of effectiveness, efficiency, and robustness to anomaly rates, data sizes, number of dimensions, anomaly patterns, and threshold settings. We also test their performance in different use cases. Finally, we provide a practical guide for detecting anomalies in time series and discussions.

CL-TAD: A Contrastive-Learning-Based Method for Time Series Anomaly Detection

Anomaly detection has gained increasing attention in recent years, but detecting anomalies in time series data remains challenging due to temporal dynamics, label scarcity, and data diversity in real-world applications. To address these challenges, we introduce a novel method for anomaly detection in time series data, called CL-TAD (Contrastive-Learning-based method for Times series Anomaly Detection), which employs a contrastive-learning-based representation learning technique. Inspired by the successes of reconstruction-based approaches and contrastive learning approaches, the proposed method seeks to leverage these approaches for time series anomaly detection. The CL-TAD method is comprised of two main components: positive sample generation and contrastive-learning-based representation learning. The former component generates positive samples by trying to reconstruct the original data from masked samples. These positive samples, in conjunction with the original data, serve as input for the contrastive-learning-based representation learning component. The representations of input original data and their masked data are used to detect anomalies later on. Experimental results have demonstrated that the CL-TAD method achieved the best performance on five datasets out of nine benchmark datasets over 10 other recent methods. By leveraging the reconstruction learning and contrastive learning techniques, our method offers a promising solution for effectively detecting anomalies in time series data by handling the issues raised by label scarcity and data diversity, delivering high performance.

Image encoding selection based on Pearson correlation coefficient for time series anomaly detection

Recently, anomaly detection in time-series data has received great attention from researchers due to its importance in problem-solving and predicting future system events. The majority of researchers combine convolutional neural networks (CNN) with image-encoding methods, including Gramian Angular Field (GAF) and Markov Transition Field (MTF). The selection of the right image-encoding method typically involves an iterative evaluation process that requires extensive computational time. This study proposes descriptive statistical analysis and Pearson correlation to find the correlation between statistical features in the encoded data from each image encoding and CNN accuracy. This information is essential for determining the image encoding method without image conversion processes and CNN trials, thus accelerating the computational time. The method tested on the initial dataset showed that the standard-deviation difference has the highest correlation with CNN accuracy (with a Pearson correlation coefficient of r = 0.944), followed by the mean difference (r = 0.705) and the count difference (r = 0.455), also resulting in a 95.35 % reduction in the computation time. The validity of this result was confirmed through the evaluation of several new datasets with different anomaly patterns, making this proposed approach suitable for selecting an appropriate image-encoding method.

Unsupervised anomaly detection by densely contrastive learning for time series data

Time series data continuously collected by different sensors play an essential role in monitoring and predicting events in many real-world applications, and anomaly detection for time series has received increasing attention during the past decades. In this paper, we propose an anomaly detection method by densely contrasting the whole time series with its sub-sequences at different timestamps in a latent space. Our approach leverages the locality property of convolutional neural networks (CNN) and integrates position embedding to effectively capture local features for sub-sequences. Simultaneously, we employ an attention mechanism to extract global features from the entire time series. By combining these local and global features, our model is trained using both instance-level contrastive learning loss and distribution-level alignment loss. Furthermore, we introduce a reconstruction loss applied to the extracted global features to prevent the potential loss of information. To validate the efficacy of our proposed technique, we conduct experiments on publicly available time-series datasets for anomaly detection. Additionally, we evaluate our method on an in-house mobile phone dataset aimed at monitoring the status of Parkinson’s disease, all within an unsupervised learning framework. Our results demonstrate the effectiveness and potential of the proposed approach in tackling anomaly detection in time series data, offering promising applications in real-world scenarios.

A novel unsupervised framework for time series data anomaly detection via spectrum decomposition

Time series is a common type of data that widely exists in various real-world scenarios, such as traffic flow data, network KPI, financial data, which can be regarded as time series. Anomaly detection in time series is an interesting research topic with a wide range of real-world applications, such as network intrusion detection, traffic situation monitoring and sensor error detection. In the real-world scenario, the frequency of anomalies is very low and little anomalous sample is available for analysis. Therefore, unsupervised methods are usually used for anomaly detection. In this paper, based on spectrum analysis and time series decomposition, an unsupervised deep framework for anomaly detection in time series data is designed. First, we decompose the original time series into trend series, seasonal series and residual series based on spectrum analysis. Then, prediction models based on long short-term memory (LSTM) networks and convolutional neural networks (CNNs) are designed to predict the trend series and seasonal series, respectively, and the residual series is reconstructed based on a Gaussian distribution. Next, the time series data are reconstructed by superimposing the predicted trend series and seasonal series and reconstructed residual series. Finally, we compare the original time series with the reconstructed time series and detect anomalies according to a certain threshold. The method proposed in this paper integrates prediction-based and reconstruction-based methods, and the experimental results on four datasets demonstrate the excellent performance of our method.

Time-frequency analysis of radon and thoron data using continuous wavelet transform

Continuous exposure to environmental radiation, whether it derives from natural or artificial sources, is thought to pose a substantial risk to public health. In addition to the health effects associated with prolonged exposure to environmental radiations, long-term measurements of these radiations can be used for a variety of beneficial purposes, such as the forecasting of impending earthquakes. Signal processing is an important application used for the purpose of forecasting. Wavelets, being signal-processing tools, are helpful in many applications such as anomaly detection in time series data. However, selection of the best wavelet for a particular application is still a problem that hasn’t found a satisfactory solution. In this study, we used continuous wavelet transform (CWT) on environmental radiations, specifically radon time series (RTS) and thoron time series (TTS) data, for the investigation of time-frequency information (TFI). The distribution of energy in the output wavelet decomposition have been investigated by several wavelet families such COIF4, DB4, SYM4 to detect frequency composition of signal and its relation with anomalies hidden in the observed data. Using discrete wavelet transform (DWT), specifically SYM4, DB4, and COIF4, we transformed the radon and thoron time series into a time-dependent sum of frequency components. Using CWT scalograms, the anomalies in the both of time series datasets (TSD) have been identified, and these anomalies have been associated with the seismic events that occurred during the period of the study. The results show that DB4 and SYM4 wavelets are good at identifying anomalies in original radon and thoron TSD, but SYM4 performs better for DWT-decomposed radon and thoron TSD.

A Method Based on an Autoencoder for Anomaly Detection in DC Motor Body Temperature

Anomaly detection has an important role in industrial systems. Abnormal situations occurring in a system cause anomalies, and the anomalies reduce system performance over time, and may also make the system malfunction. Therefore, the correct and timely detection of anomalies is of critical importance for predictive maintenance. In this study, an autoencoder-based method is proposed for anomaly detection in DC motor body temperature. The performance of the method was examined on a dataset that was created specifically for this study. In the experiments, the three-sigma outlier method was also applied on the same dataset for the same purpose and its performance results are used for comparison. The performance results of both methods are represented in terms of three measures, namely, accuracy, recall, and precision. The experimental study showed that the proposed method achieved over 96% ratios for all three measures, and it can be successfully used for anomaly detection in DC motor body temperature. Additionally, it can be concluded that the proposed system can be preferred for anomaly detection in time series data collected from different types of sensors when the performance results are taken into consideration.

Anomaly Detection on Time Series with Wasserstein GAN applied to PHM

Modern vehicles are more and more connected. For instance, in the aerospace industry, newer aircraft are already equipped with data concentrators and enough wireless connectivity to transmit sensor data collected during the whole flight to the ground, usually when the airplane is at the gate. Moreover, platforms that were not designed with such capability can be retrofitted to install devices that enable wireless data collection,as is done on Airbus A320 family. For military and heavy helicopters, HUMS (Health and Usage Monitoring System) also allows the collection of sensor data. Finally, satellites send continuously to the ground sensor data, called telemetries. Most of the time, fortunately, the platforms behave normally, faults and failures are thus rare. In order to go beyond corrective or preventive maintenance, and anticipate future faults and failures, we have to look for any drift, any change, in systems’ behavior, in data that is normal almost all the time. Moreover, collected sensor data is time series data. The problem is then anomaly detection or novelty detection in time series data. Among machine learning techniques that can be used to analyze data, Deep Learning, especially Convolutional Neural Networks, is very popular since it has surpassed human capacities for image classification and object detection. In this field, Generative Adversarial Networks are a technique to generate data similar to a potentially high dimension original dataset. In our case, generate new data could be useful to enrich the learning dataset with generated abnormal data to make it less unbalanced. Yet we are more interested in the potential of such techniques to perform anomaly detection for high dimensional data, comparing newly observed data with data that could have been generated from a distribution built from normal examples.

Exploring Techniques and Applications for Anomaly Detection in Time Series Data

Exploring Techniques and Applications for Anomaly Detection in Time Series Data

Variational autoencoder-based anomaly detection in time series data for inventory record inaccuracy

Retail companies monitor inventory stock levels regularly and manage them based on forecasted sales to sustain their market position. Inventory accuracy, defined as the difference between the warehouse stock records and the actual inventory, is critical for preventing stockouts and shortages. The root causes of inventory inaccuracy are the employee or customer theft, product damage or spoilage, and wrong shipments. In this paper, we aim at detecting inaccurate stocks of one of Turkey's largest supermarket chain using the variational autoencoder (VAE), which is an unsupervised learning method. Based on the findings, we showed that VAE is able to model the underlying probability distribution of data, regenerate the pattern from time series data, and detect anomalies. Hence, it reduces time and effort to manually label the inaccuracy in data. Since the distribution of inventory data depends on selected product/product categories, we had to use a parametric approach to handle potential differences. For individual products, we built univariate time series, whereas for product categories we built multivariate time series. The experimental results show that the proposed approaches can detect anomalies both in the low and high inventory quantities.

Anomaly Detection in Time Series Data and its Application to Semiconductor Manufacturing

Anomaly Detection in Time Series Data and its Application to Semiconductor Manufacturing

Cluster-based stability evaluation in time series data sets

In modern data analysis, time is often considered just another feature. Yet time has a special role that is regularly overlooked. Procedures are usually only designed for time-independent data and are therefore often unsuitable for the temporal aspect of the data. This is especially the case for clustering algorithms. Although there are a few evolutionary approaches for time-dependent data, the evaluation of these and therefore the selection is difficult for the user. In this paper, we present a general evaluation measure that examines clusterings with respect to their temporal stability and thus provides information about the achieved quality. For this purpose, we examine the temporal stability of time series with respect to their cluster neighbors, the temporal stability of clusters with respect to their composition, and finally conclude on the temporal stability of the entire clustering. We summarise these components in a parameter-free toolkit that we call Cluster Over-Time Stability Evaluation (CLOSE). In addition to that we present a fuzzy variant which we call FCSETS (Fuzzy Clustering Stability Evaluation of Time Series). These toolkits enable a number of advanced applications. One of these is parameter selection for any type of clustering algorithm. We demonstrate parameter selection as an example and evaluate results of classical clustering algorithms against a well-known evolutionary clustering algorithm. We then introduce a method for outlier detection in time series data based on CLOSE. We demonstrate the practicality of our approaches on three real world data sets and one generated data set.